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Naval  Applications 
of  Virtual  Reality 


The  United  States  Navy  is  focusing  its 
best  minds  and  research  laborato¬ 
ries  on  a  handful  of  VR  projects  to 
examine  the  feasibility  of  maintain¬ 
ing  or  improving  training  technologies  and 
tactical  operations.  Key  training  objectives 
are  to  compensate  for  the  educational  levels 
of  recruits,  reduced  number  of  skilled  in¬ 
structors,  and  reduced  locations  of  high- 
quality  classroom  facilities.  Additionally, 
training  should  be  more  efficient  and  affor¬ 
dable,  accommodate  varying  geographic  de¬ 
mands.  and  increase  ability  for  team  train¬ 
ing.  The  objectives  for  tactical  operations 
are  to  increase  operator  performance  by  re¬ 
ducing  workload  and  consequently  improv¬ 
ing  decision  making. 

As  with  any  new  and  emerging  technol¬ 
ogy,  VR  domain  experts  are  being  shaped 
overnight.  The  development  and  imple¬ 
mentation  of  VR  applications  require  a 
team  of  experts  with  widely  different  back¬ 
grounds.  The  Navy  has  assembled  diverse 
teams  with  elements  of  science,  art,  design, 
education,  security,  ingenuity,  and  possibly  a 


bit  of  magic. 

Navy  special  operation  missions  are  per¬ 
formed  by  SEALS  (SE-Air-Land  units)  undet 
the  direction  of  the  Naval  Special  Warfare 
Command,  These  missions  demand  exten¬ 
sive  planning  and  seamless  execution.  Foi 
the  training  of  these  forces  to  be  effective, 
attention  must  be  paid  to  the  performance 
of  the  individual  as  well  as  the  collec  tive  re¬ 
sponse  of  the  unit  A  characteristic  of  spe¬ 
cial  operations  is  the  inability  to  provide 
real-world  environments  with  the  same  last¬ 
breaking  action.  However,  VR  systems  sup¬ 
plied  with  body  function  information  could 
b-ing  these  forces  closer  to  a  believable  and 
natural  simulation. 

The  Artisan  Group  is  developing  several 
VR  conceptual  platforms  for  the  special 
operations  community  that  rouM  to  be  in¬ 
troduced  soon.  I  ne  v  k  simulations  consist 
of  the  planning,  rehearsal,  and  execution 
phases  of  a  mission.  Some  simulations  in¬ 
clude  Navy  SEAL.  Mk-VIII  swimmer  delivers 
vehicles  and  the  future  Mk-V  fast  surface  ve¬ 
hicle.  Direct  action  missions  with  high  aln- 
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Several  Naval  stenanos  require  the  re 
plat  'fluent  of  humans  In  mat  hint's  lor  haz¬ 
ardous  duties.  TIk  VR  system  makes  tele- 
operated  and  robot  it  vehicles  simpler  to 
control,  thereby  reducing  operator  stress. 
By  providing  full  immersion  audio  and  vid¬ 
eo  and  realistic  manual  control  movements 
at  the  control  site,  repair  and  maintenance 
of  ground,  air,  and  underwater  vehicles  be¬ 
comes  much  more  natural.  Full  immersion 
can  be  achieved  front  microphones  and 
cameras  positioned  on  the  remote  vehicle  or 
site.  Realistic  manual  control  can  be 
achieved  by  duplicating  the  physical  vehicle 
being  remotely  controlled  or  by  interpret- 


^opy  available  to  DDC  dbea  TV'S' 

ret  rr.it  ?  ;  -tv  i,!(.  v  T  ,  , 

Ii  il;  i  .  (lid  .  , !  .  .  •  ,■■■■■■■ 

l. u  l  ill  <  ii  ( >pl  n  .( . 

I  I  .mi  1  1  .lining  .(IK  i  U  !( iiv  ll  ill.t  i  .1 :  1  .  I  - 

ill  Ini  s  (.111  I  i<  i  V  p<  i  i  M  ,  i  \  \  k  -)  Mi  |  , 

\  idi  v  .1  mu  m  Lit  i  1 1  V" ■  1 1  g.i  mil  ig  '  1 1 1  ii '  •  : ,  c  >.  : 
im  mu  1 1  ii  1 1  m  l  ,ii  l  ii  -  i  i  .i  ii  d :  i  i.u  :t i  Ii.  ,  a  . 

V  1(1  II  Ig  lull  I  III  UK  I  'I.  ||  i  I  ll  'll  (I  I  III.',  r !  •  .  i 
ullll).  I  lie  licet  l  iol  lUc  pb\  VII  ,t  i  M  1 1  ill  i.C  I  t  ‘  -  c 
eliminated  liv  li.nkiuvt  li.md  iik!  in.dv 
movement,  tin  phvMi.il  actions  irqum-d  in 
1 1  id.  i  v  s  Slum  I.u  ol  s  i  .in  he  i  e  pin  .ucd  v\  it  j ,.  ,;ii 
the  need  loi  the  plnsu.il  mo<  kup  of  ilu 
vehicle. 

I  he  \  jvv  has  a  continuing  ; vijuu  <  mi  ni 
to  dev clop  and  test  nev\  weapon  svstem  urn 
crpls  that  could  benefit  I  rout  VR  sv  stems 
lilts  is  a  natural  extension  ol  eliminating 
the  need  ‘oi  phvsiial  simulators  and  hum  k 
ups.  New  weapons,  and  new  c  oik  ept'  loi  dt 
ploying  those  weapons  in  the  hattlelield.  <  an 
lie  simulated  m  the  virtual  wot  Id  Nets 
weapon  svstem  concepts  can  be  esploied 
without  the  expense  of  developing  pinto- 
types.  Onlookers  can  invisible  take  a  point 
of  view  from  anywhere  within  the  battle 
held  to  evaluate  tactic  s  and  strategies  Inho  • 
maiion  that  is  invisible  to  human  senses  ran 
be  examined  visuallv  or  audiblv  hi  the  sit  Di¬ 
al  world. 

Reducing  information  overload  in  time- 
critical  decision  making  is  another  kev  aiea 


Introducing  an  image  processing  architecture 
designed  for  interactive  cinema. 


One  that  makes  it 
look  like  cinema 
instead  of 
video  games  ! 


One  that  makes  virtual  reality  look  like  reality, 
except  that  reality  was  never  this  interesting.... 


VIRTUAL  SCENE  SYSTEMS 

18  WATERHOUSE  ROAD.  BOURNE.  M4  02532 
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ing  data,  mission  [nolilc  data,  and  vidi-u 
equipment  to  ivhiiiI  tlie  us,Ts  etioits. 
However,  a  Ik  lil  Mirvivahlc  3i)  rhsplav  in  an¬ 
ticipated  lor  "live-tire"  data  (olUrtiini  dur¬ 
ing  missions  to  enhance  the  situation  aware¬ 
ness  and  fast-breaking  action  lor  the  usei. 

BRAIN  WAVE  MEASUREMENT 

The  Artisan  Group  is  working  with  the 
Westinghouse  Science  and  Technologv  (.  en¬ 
ter  in  Pittsburgh.  Pa.  in  the  adaptation  of  a 
proprietary  brain  wave  measurement  tech¬ 
nology  for  use  in  VR  systems.  This  system 
will  provide  an  objective  method  for  analyz¬ 
ing  memory,  thinking,  and  attention  to  vir¬ 
tual  environments.  The  system  is  composed 
of  the  modern  principles  of  the  polygraph 
examination  with  event  related  potentials 
generated  by  the  subject’s  brain  during  VR 
immersion. 

The  brain  wave  technology  consists  of 
two  subsystems:  real-time  and  off-line.  The 
primary  function  of  the  real-time  subsystem 
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mi  nt,  data  archival  and  retiu-val.  idiisiiui 
n. hi  ul  the  test  stimuli,  d.ua  dvsis,  and 
data  display. 

To  date,  the  basic  svstem  has  been  used 
loi  training  tape  evaluations  lot  a  RADAR 
svstem,  cognitive  evaluation  lor  the  diagno¬ 
sis  of  organic  syndromes,  visual  and  heating 
testing,  and  focus  group  testing  and  product 
evaluation.  Soon,  cue  combination  studies 
of  virtual  environments  will  be  underwav 
that  will  yield  critical  data  on  stereopsis,  mo 
non,  occlusion,  texture  gradient,  long-term 
experience,  and  highlighting.  These  results 
could  yield  revolutionary  concepts  on  VR 
design  and  application. 


The  Navy  has  a  requirement 
to  develop  and  test  new 
weapon  systems  that  could 
benefit  from  VR  systems 


Let  PhotoVR  Make  Your  Worlds  Real 


Unlimited  Scene  Complexity  with 
Textures,  Shadows  and  Reflections! 

PhotoVR™  is  photorealistic  VR  software  tor  your  PC  Highly 
realistic  3-D  virtual  reality  environments  are  created  using 
AutoCAD*  or  3D-Sfudio®  and  directly  imported  into 
PhotoVR™  for  real-time  exploration.  Scenes  are  viewed  with 
sophisticated  shading  including:  multiple  lights,  textures, 
reflections,  shadows,  phong  shading,  and  bump  maps 
Typically  PhotoVR™  delivers  8  frames  per  second  for 
changing  the  user's  direction  of  view  and  3-4  frames  per 
second  for  motion  within  a  scene  This  remains  constant 
whether  a  scene  has  tO  polygons  or  100  000  polygons! 
Support  is  provided  for  HMD's,  video  tape  recorders.  NTSC 
and  VGA  monitors.  Input  devices  include  Logitech  6D. 
SpaceBafl.  Polhemus  FastTrack,  Ascension  Bird.  Trackball, 
Joystick,  and  Microsoft  Mouse.  PhotoVR™  supports  output 
on  DVI.  Spea  Fire,  and  VGA  display  cords 
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150  Mount  Bethel  Road 
Warren.  NJ  07059  USA 


TEL  (908)  580-0086 
FAX  (908)  5800092 


FIGURE  1. 

Soriobuoy  field 
coverage  (Photo 
courtesy  ol  the  U.S. 
Navy,  NCCOSC 
RDTE) 


FIGURE  2. 

Command  level 
imagery.  (Photo 
courtesy  of  the  U.S. 
Navy,  NCCOSC 
RDTE) 


TACTICAL  3D  DISPLAY 

3D  and  other  display  technologies  are  being 
studied  for  their  application  to  tactical  Na¬ 
val  operations  at  the  new  Navai  Command, 
Control  and  Ocean  Surveillance  Centei 
(NCCOSC)  as  the  Research,  Development, 
Test  and  Evaluation  Division.  Among  (he 
potential  beneficiaries  of  this  technology 
are  airborne  early  warning  and  forward  air 
control,  platform  and  force  level  battle 
management,  air  traffic  control,  compact 
flight  trainers  and  mission  planners,  and 
many  aspects  of  antisub  marine  warfare 
(ASW).  This  mission  area  is  being  investigat¬ 
ed  for  the  improvement  of  shipboard  ASW 
sensor  information  displays. 

Performance  with  displays  improves 
when  operator  workload  is  reduced,  and 
this  effect  is  amplified  by  prolonged  opera¬ 
tions.  Therefore,  displays  that  minimize  this 
workload  should  result  in  improved  opera¬ 
tor  performance  and  improved  decision 
making.  The  need,  therefore,  is  for  displays 
that  present  information  in  ways  that  take 
advantage  of  natural  human  perceptual  and 
cognitive  skills.  Few  display  systems  in  use 
today  are  more  abstract  than  those  used  to 
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over  current  systems 

An  advanced  technologv  as\v  displav  -ns 
tern  could  consist  of 

■  The  application  of  31).  stceoscopii .  Ingle 
resolution,  full  color,  helmet-mounted  .  u 
boom-mounted  displays 

■  A  3D  position  ti.ukei  (foi  the  helmet- 
mounted  displav)  and  31)  manipulators  lot 
computer  function  control 

■  The  incorporation  of  3D  audio  lot  pie- 
semation  of  multiple  beams  ftom  the  son.n 
for  intuitive  cuing  and  correlation  of  a<ous- 
tic  information  to  the  oce.m  environment 

■  Automated  speech  recognition 

■  The  intuitive  depiction  of  high-resolution 
computer-generated  imagerv  of  the  asw 
environment  based  on  integrated  displav  ol 
active,  passive,  and  environmental  sensoi 
information. 

The  exact  nature  of  the  displays  that  m.u 
prove  most  effective  are  vet  to  be  deter¬ 
mined,  but  speculation  unfettered  bv  dis¬ 
play  constraints  leads  to  some  intriguing 
possibilities. 

For  example,  in  the  active  sonar  mode  the 
3D  ASW  display  could  provide  the  operatoi 
with  a  360°  field-ol-regard  image  of  the 
ocean  bottom  topography  and  water  prop¬ 
erties  integrated  with  wave  fronts  repre¬ 
senting  the  propagation  of  the  sonar  pulses. 
The  operator  would  sec  these  wave  (routs 
reflected  off  the  local  topographs  and  all 
known  obstacles  in  that  database  and  re¬ 
fracted  by  the  model  of  the  local  water 
properties.  The  operator  would  therefore 
be  able  to  see  which  returns  are  from 
known  obstructions  and  compare  that  with 
the  actual  returns  received  bv  the  sonar.  In¬ 
tegrating  this  3D  visual  representation  with 
acoustic  analysis  and  curing  from  the  3D 
audio  system  would  give  the  operator  an  in¬ 
tuitive  picture  of  the  ASW  situation. 

Similar  display  approaches  could  be  used 
for  fixed  site  ASW  facilities.  Application  of 
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Tins  technology  could  also  he  applied  to 
ASVV  tactical  support  bv  providing  the  ship 
board  ASH’  officer  or  submarine  approach 
office!'  with  a  31)  image  of  the  ocean  envi¬ 
ronment,  the  weapons  engagement  envel¬ 
ops.  sensor  coverage  volumes,  and  the  hos¬ 
tile  submarines  that  have  been  identified.  In 
a  similar  way,  it  could  be  applied  to  the  dis¬ 
play  of  battle  management  information 
(Figure  2)  where  the  location  and  status  of 
assets  determines  the  weapons  and  sensor 
coverage. 

3D  display  technology  could  also  provide 
E-2C  controllers  with  3D  aircraft  engage¬ 
ment  information  including  launch  enve¬ 
lopes,  detection  system  coverage,  and  so  on. 
A  simpler  application  would  be  for  air  traf¬ 
fic  control  afloat  or  ashore  where  only  limit¬ 
ed  organic  sensor  information  must  be  pre¬ 
sented.  Such  a  system  might  also  benef.t 
from  the  incorporation  of  real-time  voice 
control  of  the  communications  system. 
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Illgfl  speed  gtapliKs  (omputeis  tan  gen¬ 
et  ate  aeeeptablv  realist  i<  miageis  that  is  tip 
;  dated  fast  enough  to  piesenl  an  opeiatoi 
I  with  useful  information  m  real  time.  These 
machines  are  using  multiple  processors  to 
achieve  dramatic  speed  and  cost  unpiove- 
ments  with  large  supercomputers  now  oper 
ating  with  thousands  of  processors,  Today's 
high-speed  graphics  computers  will  seem 
slow  by  the  time  a  31)  display  system  is 
ready  to  enter  the  Fleet. 

Miniature  high-resolution  displays  evolved 
from  weapons  control  and  sensor  display  re¬ 
quirements  for  military  aircraft.  Although 
the  Navy  fielded  the  first  operational  hel¬ 
met-mounted  display  (HMD),  the  most  wide¬ 
ly  used  of  these  systems  can  lx1  found  aboard 
Marine  Corps  and  Army  combat  helicop¬ 
ters.  These  systems  now  use  cathode  ray 
tubes  (CRTs)  as  small  as  0.5  inches  in  diame¬ 
ter  with  1,000  line  resolution.  Approxi¬ 
mately  1,000  line  resolution  is  required  for 
“seamless”  imagery  in  an  HMD  with  an  aver- 
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TOPES,  designed 
by  Rudy  Rucker, 
demonstrates  the 
physical  properties 
of  objects 
pictured.  Built  with 
the  Autodesk 
Ovberspace 
Developers  Kit. 
(Photo  courtesy 
of  Autodesk) 
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(inuliU'd  m  hiicat  (evolution  lo  1 1  it-  [><>1111  nl 
being  computable  with  Mandat d  k'lcMuon 
resolution  (1(1)  1 1 n<i i>t-  height  ol  48(1  pixels 
vs  .")(>(>  lines  ol  r.iMci  sail  v  ideo  lot  com  rn- 
tional  TV)  Although  tins  is  a  great  im¬ 
provement  met  the  previous  generation  ol 
240  pixel-high  LCDs,  tins  resolution  is  still 
low  enough  that  the  image  appears  verv 
grains  when  magnified  as  part  ol  a  stereo¬ 
scopic  dispiav  svstem.  fortunately.  higher 
resolution  color  LCDs  are  likely  to  emerge 
on  the  commercial  market  within  five  years 
because  of'  a  Defense  Advanced  Research 
Projects  Agency  (I)ARl’A)  funded  research 
effort  at  the  David  Sarnoff  Research 
Center. 

The  most  prevalent  3D  position  sensoi 
system  todav  uses  a  magnetic  field  gener¬ 
ator  affixed  to  'he  local  environment  and  a 
field  sensor  attached  10  the  device  to  be 
monitored.  A  computer  system  monitors 
the  field  sensoi  and  calculates  the  position 
and  attitude  of  the  setisor  based  on  the  na¬ 
ture  of  the  portion  of  the  magnetic  field  in 
which  it  is  immersed.  These  units  are  small 
and  rugged,  but  they  are  sensitive  to  the 
presence  of  ferrous  metals  that  can  distort 
the  magnetic  field  geometry.  Work  is  being 
done  to  reduce  this  problem  and  to  speed 
up  the  operation  of  the  system.  Other  posi¬ 
tion  trackers  are  now  reaching  the  commer¬ 
cial  market,  however.  One  unit  uses  multi¬ 
ple  ultrasonic  sensors  and  sources  to 
monitor  position  and  attitude.  An  analo¬ 
gous  multi-element  infrared  system  is  also 
available.  These  systems  are  not  sensitive  to 
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i  i  rans  1 1  ,i  in  1  iiiic a  ion  iu<  m  p<u  ate  i  into  >  mu 
|  it v.  would  seem  to  have  emanated  horn  am 
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|  ter  a  sound  so  that  it  would  seem  to  have 
come  from  am  chosen  local  mu  m  spaa-  ex¬ 
ternal  to  the  listener.  Recent  v-.oik  at  Na¬ 
tional  Aeronautics  and  Space  A< Immist i a- 
tion  (NASA)  Ames  Researc  h  Uentei  tesuhed 
m  these  transforms  being  me  or  pm  alec!  into 
digital  circuitry  and  manied  to  cotnputeis 
that  could  sinutlt aneouslv  inomtoi  the  mi- 
dilation  ol  the  operatot  s  head  'using  a  31) 
position  tracker)  and  transform  a  sound  so 
that  it  would  seem  to  he  station. in  while  the 
operatot  moved. 

THE  F  UTURE 

These  technologies  are  alrc.idv  being  ap 
plied  to  numerous  civilian  tasks  m<  hiding 
aichitecture:  mechanical  design  in  the  auto¬ 
motive  and  aerospace  industries;  pharma¬ 
ceutical  research;  medical  imagerv  (or  3D 
display  of  CAT  scan.  NMR,  and  ultrasonic 
medical  data;  education:  emei  tamme  nt .  and 
functional  aids  for  the  handicapped  1 1  these 
new  technologies  are  properlv  applied  to 
the  tactical  Naval  environment,  their  impact 
on  operational  effectiveness  and  tt. lining  ; 
methods  and  systems  could  he  dramatic. 

History  has  taught  us  that  the  require¬ 
ment  to  operate  a  global  Naw  will  not  di¬ 
minish  at  a  rate  commensurate  with  the  de¬ 
crease  in  our  assets  Budgets  and  stall  levels  < 
will  probably  continue  to  decrease  over  the  1 
next  decade  or  two.  Therefore  the  work-  I 
load  for  each  person  will  increase,  and  the  i 
tactical  cost  of  losing  one  platform  out  of  a  j 
reduced  inventory  will  be  magnified.  Sys¬ 
tems  that  can  improve  operational  perfor-  i 
mance  while  requiring  lower  staffing  will  be  j 
critical,  and  those  systems  that  can  achieve  j 
the  greatest  performance  improvements  at 
the  lowest  cost  will  be  essential  v-  j 
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